United  States  General  Accounting  Office 


GAO 


Report  to  Congressional  Addressees 


2000  CENSUS 

Refinements  to  Full 
Count  Review 
Program  Could 
Improve  Future  Data 
Quality 


GAO 

Accountability  *  Integrity  *  Reliability 


GAO-02-562 


Report  Documentation  Page 


Report  Date 

Report  Type 

Dates  Covered  (from...  to) 

00JUL2002 

N/A 

- 

Title  and  Subtitle 

Contract  Number 

2000  CENSUS:  Rermements  to  Full  Count  Review 
Program  Could  Improve  Future  Data  Quality 

Grant  Number 

Program  Element  Number 

Author(s) 

Project  Number 

Task  Number 

Work  Unit  Number 

Performing  Organization  Name(s)  and  Address(es) 

U.S.  General  Accounting  Office  441  G  Street  NW, 

Room  FM  Washington,  D.C.  20548 

Performing  Organization  Report  Number 

GAO-02-562 

Sponsoring/Monitoring  Agency  Name(s)  and 

Sponsor/Monitor’s  Acronym(s) 

Address(es) 

Sponsor/Monitor’s  Report  Number(s) 

Distribution/Availability  Statement 

Approved  for  public  release,  distribution  unlimited 

Supplementary  Notes 

Abstract 

see  report 

Subject  Terms 

Report  Classification 

unclassified 

Classification  of  this  page 

unclassified 

Classification  of  Abstract 

unclassified 

Limitation  of  Abstract 

SAR 

Number  of  Pages 

23 


Contents 


Letter  1 

Results  in  Brief  2 

Scope  and  Methodology  4 

Background  5 

The  Numbers  and  Kinds  of  Issues  Identified  During  Full  Coimt 

Review  6 

The  Bureau  Resolved  Few  Issues  Prior  to  Releasing  Public  Law 

Data  8 

Lessons  Learned  That  Can  Inform  Future  Data  Clearance  Reviews  9 

Conclusions  12 

Recommendations  for  Executive  Action  13 

Agency  Comments  and  Our  Evaluation  13 


Appendix  I  Disposition  of  Data  Issues  Identified  During  Full 

Count  Review  by  State  15 


Appendix  II  Comments  From  the  Department  of  Commerce  17 


Appendix  III  GAO  Products  on  the  Results  of  the  2000  Census  and 

Lessons  Learned  for  a  More  Cost-Effective  Census  in 
2010  20 


Tables 


Table  1:  Types  of  Issues  Flagged  During  Full  Count  Review  7 

Table  2:  Disposition  of  Data  Issues  Identified  During  Full  Count 

Review  9 


Page  i 


GAO-02-562  2000  Census 


i 

GAO 

^^^^^^^^^Accountability  *  Integrity  *  Reliability _ 

United  States  General  Acconnting  Office 
Washington,  DC  20548 


July  3,  2002 

The  Honorable  Henry  A.  Waxman 
Ranking  Minority  Member 
Committee  on  Government  Reform 
House  of  Representatives 

The  Honorable  William  Lacy  Clay,  Jr. 

The  Honorable  Carolyn  B.  Maloney 
House  of  Representatives 

Demographic  Full  Count  Review  was  one  of  a  number  of  quality  assurance 
programs  the  Bureau  of  the  Census  developed  for  the  2000  decennial 
headcount  to  help  ensure  the  completeness  and  accuracy  of  census  data. 
Analysts  were  to  identify,  investigate,  and  document  suspected  data 
discrepancies  or  “issues”  in  order  to  clear  census  data  files  and  products 
for  subsequent  processing  or  public  release.  Bureau  reviewers  were  to 
determine  whether  and  how  to  correct  the  data  by  weighing  quality 
improvements  against  time  and  budget  constraints. 

According  to  bureau  officials,  because  the  bureau  lacked  sufficient  staff  to 
conduct  Full  Count  Review  on  its  own,  it  contracted  out  some  of  the 
analysts’  work  to  members  of  the  Federal-State  Cooperative  Program  for 
Population  Estimates  (FSCPE),  an  organization  composed  of  state 
demographers  that,  since  its  inception  in  1973,  has  worked  with  the 
bureau  to  ensure  accurate  state  and  local  population  estimates.  The 
bureau  believed  that  FSCPE  members’  knowledge  of  the  demographic 
characteristics  of  their  respective  states  could  help  the  bureau  review  data 
files  and  products,  including  politically  sensitive  apportionment  and 
redistricting  data  files  known  as  public  law  data.^ 

This  letter  responds  to  your  request  to  review  the  FSCPE  members’ 
participation  in  the  Full  Count  Review  program.  As  agreed  with  your 


^  The  Census  Act  (13  USC  sec.  141(a,b))  requires  the  Secretary  of  Commerce  to  deliver 
state  population  counts  to  the  President  within  9  months  after  the  census  date  (for  the  2000 
Census,  this  meant  no  later  than  December  31,  2000).  State  population  counts  are  used  to 
reapportion  seats  in  the  U.S.  House  of  Representatives.  The  Census  Act  also  requires  the 
Secretary  of  Commerce  to  send  census  population  tabulations  to  the  states  no  later  than  1 
year  after  the  April  1  decennial  census  date  (13  USC  sec.  141  (c)).  These  numbers  are  used 
for  redistricting. 
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offices,  we  provide  information  on  (1)  the  number  and  kinds  of  data  issues 
identified  by  FSCPE  members  and  bureau  analysts,  (2)  how  the  bureau 
used  the  information  developed  during  Full  Count  Review,  and  (3)  lessons 
learned  from  the  conduct  of  Full  Count  Review  that  can  help  the  bureau 
plan  for  the  2010  Census.  This  report  is  the  latest  in  a  series  of  reviews  that 
examine  the  results  of  key  census-taking  operations  and  highlight 
opportunities  for  reform  (see  app.  Ill  for  a  list  of  products  issued  to  date). 


Results  in  Brief 


FSCPE  members  documented  1,402  data  issues,  which  was  about  29 
percent  of  the  4,809  issues  identified  by  both  FSCPE  and  bureau  analysts 
during  Full  Count  Review.  Of  the  4,809  issues,  1,599  (33  percent)  dealt 
with  “group  quarters,”  where  the  location  or  population  counts  for  prisons, 
nursing  homes,  dormitories,  and  other  group  living  facilities  differed  from 
what  analysts  expected.  Of  the  1,599  group  quarters  issues,  FSCPE 
members  identified  567  (35  percent).  Discrepancies  relating  to  housing 
unit  counts,  population  data,  and  demographic  characteristics  accounted 
for  1,150  issues  (24  percent),  375  of  which  (33  percent)  were  identified  by 
FSCPE  members.  The  bureau  was  unable  to  classify  2,060  issues  (43 
percent)  because  of  insufficient  documentation. 

According  to  bureau  officials,  5  of  the  4,809  issues  identified  during  Full 
Count  Review  were  corrected  prior  to  the  December  31,  2000,  release  of 
apportionment  data,  and  the  April  1,  2001,  release  of  redistricting  data.  All 
five  were  group  quarters  issues  where  the  bureau  had  the  correct 
population  count  for  each  facility  but  placed  them  in  the  wrong  locations. 
The  “misplaced”  group  quarters  included  a  military  base,  a  federal  medical 
center,  and  multiple  facilities  at  two  prisons  and  a  college.  The  bureau  was 
able  to  correct  these  issues  because  FSCPE  members  identified  them  early 
in  the  Full  Count  Review  process  and  thoroughly  documented  them  so 
that  they  did  not  require  additional  research  or  field  verification.  Because 
the  bureau  lacked  the  time  and  field  staff  from  its  regional  offices  to  do 
any  further  investigative  work,  the  4,804  remaining  issues  went  unresolved 
prior  to  the  release  of  the  redistricting  data.  As  a  result,  uncertainties 
surround  the  accuracy  of  the  census  data  for  the  affected  localities. 

Overall,  of  the  4,809  issues  identified  during  Full  Count  Review,  4,267  (89 
percent)  were  not  subjected  to  further  investigation  by  the  bureau  in  large 
part  because  of  insufficient  documentation.  The  bureau  plans  to  review 
537  issues  as  part  of  a  subsequent  effort  called  the  Count  Question 
Resolution  (CQR)  program,  which  the  bureau  developed  to  respond  to 
challenges  to  the  census  data  brought  by  state,  local,  or  tribal  governments 
(see  app.  1  for  the  disposition  of  Full  Count  Review  data  issues  by  state). 
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The  bureau’s  preliminary  plans  for  the  2010  Census  include  a  Full  Count 
Review  program.  Our  examination  identified  several  areas  where 
improvements  are  warranted.  Foremost  among  these  is  that  it  will  be 
important  for  the  bureau  to  investigate  and  resolve  a  larger  number  of 
issues  prior  to  the  release  of  the  public  law  data.  We  found  three  factors 
that  limited  the  bureau’s  ability  to  do  so  for  the  2000  Census.  First,  bureau 
officials  said  that  correcting  individual  issues  was  beyond  the  scope  of  the 
Full  Count  Review  program.  They  noted  that  the  program  was  developed 
in  February  1999,  just  14  months  prior  to  Census  Day  2000  and,  as  a  result, 
the  bureau  was  imable  to  test  the  program,  or  integrate  it  with  other 
census  operations  and  units  that  could  have  investigated  the  issues  and 
made  corrections. 

Second,  the  bureau’s  requirements  for  documenting  data  issues  were  not 
clearly  defined.  As  a  result,  there  was  considerable  variation  in  the  quality 
of  the  documentation  analysts  used  to  support  issues,  and  in  a  number  of 
cases,  the  bureau  had  difficulty  determining  the  precise  nature  of  an  issue, 
or  if  in  fact  an  issue  even  existed. 

A  third,  and  related  item  that  limited  the  bureau’s  ability  to  resolve  a  larger 
number  of  issues  was  the  fact  that  the  bureau  had  no  mechanism  for 
managing  its  workload.  Unlike  the  CQR  program  where  the  bureau 
requires  specific  documentation  before  committing  resources  to 
investigate  local  issues,  the  Full  Count  Review  program  had  no  filter  for 
screening  submissions.  Had  the  bureau  first  categorized  issues  based  on 
the  quality  and  precision  of  the  documentation  provided,  the  bureau  could 
have  prioritized  its  investigative  workload  and  perhaps  reconciled  a  larger 
number  of  data  issues. 

Another  area  where  there  is  room  for  improvement  concerns  the 
consistency  and  clarity  of  the  Full  Count  Review  program’s  objectives.  For 
example,  training  materials  noted  that  one  purpose  of  the  Full  Count 
Review  program  was  to  document  issues  and  “fix  what  can  be  fixed.” 
However,  this  conflicts  with  statements  from  bureau  managers  that 
correcting  issues  was  outside  the  scope  of  the  Full  Count  Review  program. 
The  different  messages  may  have  raised  FSCPE  members’  expectations 
that  the  bureau  was  going  to  correct  a  larger  number  of  data  issues  than  it 
actually  did. 

That  the  apportionment  and  redistricting  data  were  released  with  around 
4,800  unresolved  data  issues  of  unknown  validity,  magnitude,  and  impact 
is  cause  for  concern.  The  bureau  did  not  fully  exploit  the  Full  Count 
Review  program’s  potential,  and  missed  an  opportunity  to  verify  and 
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possibly  improve  the  quality  of  the  public  law  data.  To  help  develop  the 
Full  Count  Review  program  into  a  more  effective  tool  for  enhancing  data 
quality  in  the  future,  as  well  as  to  make  better  use  of  FSCPE  members’ 
input,  we  recommend  that  the  Secretary  of  Commerce  explore  ways  of 
reconciling  a  larger  number  of  data  issues  prior  to  the  release  of  public 
law  data.  Specific  steps  include 

(1)  planning  the  Full  Count  Review  program  early  in  the  census  cycle  and 
testing  procedures  under  conditions  as  close  to  the  actual  census  as 
possible, 

(2)  integrating  the  Full  Coimt  Review  program  with  other  census 
organizational  units  and  operations  to  ensure  the  bureau  has  sufficient 
time  and  field  support  to  investigate  issues, 

(3)  developing  clear  guidelines  on  the  minimum  documentation  needed  for 
the  bureau  to  investigate  individual  data  issues, 

(4)  categorizing  issues  on  the  basis  of  the  quality  and  precision  of  the 
documentation,  and  investigating  first  those  issues  that  are  best 
documented  and  thus  more  easily  resolved,  and 

(5)  exploring  the  feasibility  of  using  staff  from  the  bureau’s  regional 
offices  to  help  investigate  data  issues  in  the  field  prior  to  the  release  of 
public  law  data. 

The  Secretary  of  Commerce  should  also  ensure  that  the  bureau  clarifies 
and  consistently  communicates  to  FSCPE  members  the  objectives  of  the 
Full  Count  Review  program  and  how  the  bureau  plans  to  use  the 
information  derived  from  it. 

The  Secretary  of  Commerce  forwarded  written  comments  from  the  Bureau 
of  the  Census  on  a  draft  of  this  report  (see  app.  II).  The  bureau  concurred 
with  all  of  our  recommendations  and  had  no  comments  on  them.  The 
bureau  also  provided  minor  technical  corrections  that  we  incorporated  in 
our  report  as  appropriate. 


Scope  and 
Methodology 


To  obtain  information  on  the  number  and  kinds  of  issues  identified  by  the 
FSCPE  and  bureau  analysts  and  to  determine  how  the  bureau  used  the 
information  developed  during  the  Full  Count  Review  program,  we 
analyzed  the  work  papers  submitted  by  FSCPE  members  and  other 
participants  in  the  Full  Count  Review  program.  We  also  analyzed  data 
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from  the  bureau’s  Count  Review  Information  System,  a  database  that  the 
bureau  used  to  track  issues  flagged  during  the  review  process.  We  did  not 
independently  verify  the  information  it  contained. 

To  identify  lessons  learned  for  future  improvements,  we  examined  bureau 
training  manuals,  statements  of  work,  process  models,  and  other 
documents  that  described  the  objectives,  processes,  and  decision-making 
criteria.  We  also  reviewed  the  results  of  a  survey  the  bureau  conducted  of 
FSCPE  members  that  asked  them  to  rate  their  experience  with  Full  Count 
Review  processes  and  tools,  bureau  staff,  and  the  overall  effectiveness  of 
the  Full  Count  Review  program.  In  addition,  we  interviewed  managers  in 
the  bureau’s  Population  Division  and  other  officials  responsible  for 
implementing  the  Full  Count  Review  program,  as  well  as  three  FSCPE 
members. 

We  performed  our  audit  in  Washington,  D.C.,  and  the  bureau’s 
headquarters  in  Suitland,  Maryland,  between  May  2001  and  April  2002.  Our 
work  was  done  in  accordance  with  generally  accepted  government 
auditing  standards. 

On  April  26,  2002,  we  requested  comments  on  a  draft  of  this  report  from 
the  Secretary  of  Commerce.  The  Secretary  forwarded  the  bureau’s  written 
comments  on  Jime  11,  2002  (see  app.  II).  We  address  them  in  the  “Agency 
Comments  and  Evaluation”  section  of  this  report. 


Background 


Accurate  census  results  are  critical  because  the  data  are  used  to 
reapportion  seats  in  the  House  of  Representatives  and  for  congressional 
redistricting.  Moreover,  census  data  remain  an  important  element  in 
allocating  federal  aid  to  state  and  local  governments.  With  billions  of 
dollars  at  stake,  the  data  are  scrutinized  intensely  for  accuracy. 

To  help  ensure  the  accuracy  of  census  data,  the  bureau  conducted  a 
number  of  quality  assurance  programs  throughout  the  course  of  the 
census.  One  such  program  was  the  Full  Coimt  Review  program,  which  was 
designed  to  rapidly  examine,  rectify  if  possible,  and  clear  census  data  files 
and  products  for  subsequent  processing  or  public  release.  The  bureau 
expected  data  analysts  to  identify  data  discrepancies,  anomalies,  and  other 
data  “issues”  by  checking  the  data  for  its  overall  reasonableness,  as  well  as 
for  its  consistency  with  historical  and  demographic  data,  and  other  census 
data  products.  The  Full  Count  Review  program  ran  from  June  2000 
through  March  2001. 
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According  to  bureau  officials,  because  the  bureau  could  not  complete  the 
Full  Count  Review  workload  without  a  costly  staff  increase,  some  of  the 
analysts’  work  was  contracted  to  members  of  the  FSCPE,  an  organization 
composed  of  state  demographers  that  works  with  the  bureau  to  ensure 
accurate  state  and  local  population  estimates. 

The  bureau  contracted  with  53  FSCPE  members  who  reviewed  data  for  39 
states  and  Puerto  Rico.  Bureau  employees  reviewed  data  for  the  11 
remaining  states  and  the  District  of  Columbia  without  FSCPE 
representation  in  Full  Count  Review.  Bureau  and  FSCPE  analysts  were  to 
ensure  that  (1)  group  quarters  were  correctly  placed  or  “geocoded”  on 
census  maps,  and  that  their  population  counts  and  demographic 
characteristics  appeared  reasonable  and  (2)  population  counts  of  other 
areas  were  in  line  with  population  estimates.  They  were  to  describe  each 
issue  flagged  and  provide  supporting  documentation  derived  from  bureau 
resources  and/or  resources  of  the  respective  state  government. 
Additionally,  bureau  officials  stated  that  staff  from  the  regional  offices 
reviewed  demographic  data  from  the  50  states,  Puerto  Rico,  and  the 
District  of  Columbia.  They  focused  on  identifying  inconsistent 
demographic  characteristics  and  did  not  necessarily  concentrate  on  any 
one  particular  state  or  locality.  The  bureau  reimbursed  state  governments 
for  wages  and  expenses  FSCPE  members  incurred. 

A  separate  set  of  employees  from  the  bureau’s  Population  Division 
assessed  issues  identified  by  Full  Count  Review  analysts  based  on  (1)  the 
adequacy  of  the  documentation  supporting  each  issue,  and  (2)  whether  or 
not  they  believed  the  issue  to  be  resolvable  through  follow-up  research  by 
the  bureau.  Those  issues  deemed  to  have  adequate  documentation  were 
classified  as  a  “group  quarters,”  “housing  unit,”  or  “household”  or  “other” 
issue.  Bureau  officials  told  us  that  the  remaining  issues  could  not  be 
categorized  because  the  nature  of  the  issue  could  not  be  determined  from 
the  documentation. 


The  Numbers  and 
Kinds  of  Issues 
Identified  During  Full 
Count  Review 


Bureau  data  show  that  after  reviewing  census  data  for  39  states  and  Puerto 
Rico,  FSCPE  members  identified  a  total  of  1,402  issues,  or  about  29 
percent  of  the  4,809  issues  collectively  flagged  during  Full  Count  Review 
(see  table  1).  Since  the  bureau  has  yet  to  resolve  most  of  these  issues,  it  is 
not  known  whether  they  are  necessarily  errors. 
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Table  1 ;  Types  of  Issues  Flagged  During  Full  Count  Review 

Type  of  Issue 

Issues  Identified  by 
FSCPE  members 

Issues  identified  by 
bureau  personnel 

Total 

Group  quarters 

567 

1,032 

1,599 

Housing  unit 

203 

276 

479 

Household 

134 

154 

288 

Other 

38 

345 

383 

No  type  assigned 

460 

1,600 

2,060 

Total 

1,402 

3,407 

4,809 

Note:  201  issues  were  placed  in  more  than  one  category. 


Source:  GAO  analysis  of  U.S.  Census  Bureau  data. 


Table  1  also  shows  that  group  quarters  issues  were  those  most  frequently 
identified  by  the  bureau,  accounting  for  1,599  of  the  4,809  issues  identified 
(33  percent).  Group  quarters  issues  relate  to  suspected  discrepancies  in 
the  population  counts  and  locations  of  prisons,  dormitories,  nursing 
homes,  and  similar  group  living  arrangements.  Analysts  also  identified  479 
housing  unit  issues  (10  percent  of  the  total),  and  288  household  issues  (6 
percent  of  the  total).  With  housing  unit  issues,  the  count  of  occupied 
housing  units  differed  from  what  analysts  expected  while  household 
issues  had  population  data  for  occupied  residences  that  differed  from 
what  analysts  expected.  There  were  also  383  issues  (8  percent)  that  the 
bureau  classified  as  “other”.  They  contained  questions  concerning  the 
demographic  characteristics  of  the  data  such  as  age,  race,  and  gender.  The 
bureau  was  unable  to  classify  2,060  issues  (43  percent).  Bureau  officials 
told  us  that  in  these  cases,  analysts  did  not  provide  sufficient 
documentation  for  the  bureau  to  determine  the  nature  of  the  issue. 

According  to  bureau  officials,  bureau  analysts  identified  a  larger  number 
of  issues  than  FSCPE  members — and  a  far  larger  number  of  issues  for 
which  the  bureau  could  not  assign  a  type — ^because  bureau  analysts  used 
an  automated  process  that  compared  data  from  the  2000  Census  to 
independent  benchmarks  such  as  the  1990  Census,  and  flagged  any 
anomalies.  This  process  alerted  bureau  officials  that  there  were  data 
discrepancies,  but  did  not  indicate  their  nature.  By  comparison,  FSCPE 
members  compared  census  data  to  administrative  records  and  other  data, 
and  were  better  able  to  document  specific  issues. 

Examples  of  the  three  issue  categories  and  how  they  were  found  include: 

Group  quarters  issues:  Analysts  noticed  that  the  group  quarters  population 
count  in  a  particular  census  tract  of  a  large  midwestem  city  appeared  to 
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be  too  high,  while  a  neighboring  tract  had  a  correspondingly  low  group 
quarters  population  count.  By  comparing  state  administrative  records  to 
information  obtained  from  bureau  resources,  analysts  determined  that 
bureau  data  had  placed  college  dormitories  in  the  wrong  tract. 

Housing  unit  issues:  An  urban  area  had  a  large  amount  of  redevelopment 
since  the  1990  census.  As  part  of  this,  several  condominiums  and 
apartment  complexes  were  built  which  substantially  increased  the  number 
of  housing  units  in  a  particular  census  tract.  However,  when  the  analyst 
compared  population  data  from  the  1990  Census  and  2000  Census,  the 
2000  Census  did  not  appear  to  reflect  this  increase,  and  it  was  flagged. 

Household  issues:  Data  from  the  2000  Census  appeared  to  accurately 
reflect  the  large  amount  of  new  house  construction  that  had  taken  place 
within  a  specific  census  tract.  However,  because  the  population  count 
differed  from  that  indicated  by  other  data  sources,  the  analyst  flagged  it  as 
an  issue  to  avoid  undercounting  the  population. 


The  Bureau  Resolved 
Few  Issues  Prior  to 
Releasing  Public  Law 
Data 


Bureau  officials  told  us  that  they  used  the  Full  Count  Review  program  to 
identify  systemic  errors  such  as  those  that  could  be  produced  by  software 
problems.  None  were  found.  The  officials  noted  that  the  bureau  generally 
did  not  use  the  Full  Count  Review  program  to  resolve  individual  issues. 
According  to  bureau  officials,  the  bureau  corrected  data  for  5  of  the  4,809 
issues  prior  to  the  December  31,  2000,  release  of  reapportionment  data 
and  the  April  1,  2001,  release  of  redistricting  data. 


According  to  bureau  officials,  FSCPE  members  identified  the  five  issues, 
all  of  which  involved  group  quarters  that  were  placed  in  the  wrong 
locations,  but  the  population  counts  were  correct.  They  included  (1)  a 
military  base  in  Nevada,  (2)  10  facilities  at  a  college  in  Wisconsin,  (3)  9 
facilities  at  a  prison  in  New  York  City,  (4)  14  facilities  at  a  Washington 
prison,  and  (5)  a  federal  medical  center  in  Massachusetts. 


Bureau  officials  said  that  the  bureau  was  able  to  correct  these  issues  for 
two  reasons.  First,  FSCPE  analysts  found  them  early  in  the  Full  Count 
Review  program,  while  the  bureau  was  processing  a  key  geographic  data 
file  and  was  thus  able  to  incorporate  the  corrections  before  the  data  were 
finalized.  Second,  the  FSCPE  analysts  had  thoroughly  documented  the 
issues  and  recommended  how  the  bureau  should  correct  the  errors.  The 
five  errors  did  not  require  additional  research  or  field  verification. 
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Bureau  officials  told  us  that  they  lacked  the  time  to  research  the  remaining 
issues,  as  well  as  field  staff  to  inspect  purported  discrepancies  prior  to  the 
release  of  the  public  law  data.  As  a  result,  the  bureau  missed  an  important 
opportunity  to  verify  and  possibly  improve  the  quality  of  the  data,  and 
instead  the  apportionment  and  redistricting  data  were  released  with  more 
than  4,800  unresolved  issues.  Until  these  issues  are  resolved,  uncertainties 
will  surround  the  accuracy  of  the  census  data  for  the  affected  localities. 

Some  of  the  issues  might  be  resolved  under  the  CQR  program,  which  the 
bureau  designed  to  respond  to  challenges  to  housing  unit  and  group 
quarters  population  counts  received  from  state,  local,  or  tribal 
governments.^  However,  as  shown  in  table  2,  of  the  4,804  issues  remaining 
after  Full  Count  Review,  1,994  (42  percent)  were  referred  to  CQR,  and  of 
these,  537  (11  percent)  were  accepted  for  further  investigation.  The 
remaining  1,457  issues  referred  to  CQR  did  not  meet  the  bureau’s 
documentation  requirements  and  consequently,  the  bureau  took  no  further 
action  on  them  (see  app.  1  for  the  disposition  of  Full  Count  Review  data 
issues  by  state). 


Table  2:  Disposition  of  Data  Issues  Identified  During  Full  Count  Review 

Action 

Number  of  issues 

Identified  during  Full  Count  Review 

4,809 

Corrected  during  Full  Count  Review 

5 

Identified  during  Full  Count  Review  but  not  referred  to  CQR 

2,810 

Referred  to  CQR 

1,994 

Accepted  by  CQR 

537 

Rejected  by  CQR 

1,457 

Source:  GAO  analysis  of  U.S.  Census  Bureau  data. 


Lessons  Learned  That 
Can  Inform  Future 
Data  Clearance 
Reviews 


The  overall  results  of  the  Full  Coimt  Review  program  and  FSCPE 
members’  participation  appear  to  be  mixed.  On  the  one  hand,  the  bureau 
reported  that  the  Full  Count  Review  program  was  successful  in  that  it  met 
a  number  of  performance  goals.  For  example,  the  bureau  reported  that  the 
Full  Count  Review  program  was  comprehensive  in  its  review  of  geography 
and  content,  and  was  completed  in  time  to  release  the  public  law  data  on 
schedule. 


^  CQR  began  in  late  June  2001,  and  is  scheduled  to  end  in  September  2003. 
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Moreover,  between  January  and  February  2001,  the  bureau  surveyed  the 
40  entities  that  participated  in  Full  Count  Review  and  the  results  suggest 
that  most  FSCPE  members  were  satisfied  with  their  Full  Count  Review 
experience.  For  example,  respondents  indicated  that  they  were  generally 
satisfied  with  such  aspects  of  the  program  as  its  processes  and  technical 
tools,  bureau  staff,  and  the  overall  effectiveness  of  the  review  in  terms  of 
positioning  states  to  use  and  understand  census  data.  In  addition,  bureau 
officials  believe  the  Full  Count  Review  program  benefited  from  FSCPE 
members’  local  demographic  knowledge. 

Nevertheless,  our  review  of  the  Full  Count  Review  program  highlighted 
several  areas  where  there  is  room  for  future  improvement.  It  will  be 
important  for  the  bureau  to  address  these  shortcomings  as  its  preliminary 
plans  call  for  a  similar  operation  as  part  of  the  2010  Census.  According  to 
bureau  officials,  the  bureau  plans  to  include  a  Full  Count  Review  program 
in  census  tests  it  expects  to  conduct  later  in  the  decade. 

Foremost  among  the  areas  in  need  of  improvement  is  resolving,  to  the 
extent  practical,  a  larger  number  of  data  issues  prior  to  the  release  of 
apportionment  data  by  December  31  of  the  census  year,  and  redistricting 
data  by  April  1  of  the  following  year.  We  found  three  factors  that  limited 
the  bureau’s  ability  to  do  so. 

First,  according  to  bureau  officials,  resolving  individual  issues  was  outside 
the  scope  of  the  Full  Count  Review  program.  They  explained  that  the 
program  was  poorly  integrated  with  other  census  operations  and  units  that 
could  have  investigated  the  issues  and  corrected  the  data  if  warranted. 

This  was  because  the  Full  Count  Review  program,  with  FSCPE 
participation,  was  not  conceived  until  February  1999,  which  was 
extremely  late  in  the  census  cycle,  coming  just  14  months  before  Census 
Day,  April  1,  2000. 

The  timing  of  the  decision  stemmed  from  the  Supreme  Court’s  January 
1999  ruling^  that  prohibited  the  bureau  from  using  statistical  sampling  for 
purposes  of  congressional  apportionment  (the  bureau  originally  planned  a 
“one-number”  census  that  would  have  integrated  the  results  of  a  sample 
survey  with  the  traditional  census  to  provide  one  adjusted  set  of  census 
numbers).  Faced  with  the  larger  workload  of  reviewing  two  sets  of  data — 
adjusted  and  unadjusted — ^the  bureau  decided  to  enlist  the  help  of  FSCPE 


’^Department  of  Commerce  v.  United  States  House  of  Representatives,  625  U.S.  316  (1999). 
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members  in  order  to  meet  the  deadlines  for  releasing  the  public  law  data. 
Additionally,  the  bureau’s  decision  came  after  the  1998  dress  rehearsal  for 
the  2000  Census,  which  meant  that  the  bureau  had  no  opportunity  to  test 
the  Full  Count  Review  program  in  an  operational  environment. 

Bureau  officials  explained  that  if  more  time  or  staff  were  available  in  the 
future,  it  would  be  possible  to  correct  a  larger  number  of  individual  issues 
prior  to  the  release  of  the  public  law  data.  They  noted  that  field  staff 
would  be  needed  to  help  verify  issues,  and  the  effort  would  require  close 
coordination  with  several  bureau  units. 

A  second  factor  that  affected  the  bureau’s  ability  to  correct  a  larger 
number  of  issues  was  that  the  bureau’s  requirements  for  documenting  data 
issues  were  not  clearly  defined.  For  example,  the  training  materials  we 
examined  did  not  provide  any  specific  guidance  on  the  type  of  evidence 
analysts  needed  to  support  data  issues.  Instead,  the  training  materials  told 
analysts  to  supply  as  much  supporting  information  as  necessary.  This 
could  help  explain  the  variation  that  we  observed  in  the  quality  of  the 
documentation  analysts  provided.  Indeed,  while  some  analysts  provided 
only  minimal  data,  others  supported  issues  with  state  and  local 
administrative  records,  historical  data,  photographs,  and  maps.  In  some 
cases,  the  bureau  had  difficulty  determining  the  precise  nature  of  an  issue 
or  if  in  fact  an  issue  even  existed. 

In  contrast,  the  CQR  program  provides  comprehensive  guidelines  on  the 
documentation  required  for  making  submissions.  The  guidance  available 
on  the  bureau’s  CQR  web  site  notes  that  before  the  bureau  will  investigate 
concerns  raised  by  government  and  tribal  officials,  such  officials  must  first 
supply  specific  information.  The  guidance  then  details  the  information 
needed  to  support  boundary  corrections,  geocoding  and  coverage 
corrections,  and  group  quarters  population  corrections. 

A  third,  and  related  factor  that  affected  the  bureau’s  ability  to  resolve  a 
larger  number  of  issues  stemmed  from  the  fact  that  the  bureau  had  no 
mechanism  for  managing  the  Full  Count  Review  workload.  Unlike  the 
CQR  program,  where  the  bureau  required  local  governments  to  provide 
specific  documentation  before  it  would  commit  resources  to  investigate 
local  data  issues,  the  Full  Count  Review  program  had  no  filter  for 
screening  submissions  based  on  the  quality  of  the  documentation.  Better 
guidance  on  documenting  issues  for  the  Full  Count  Review  program  could 
make  the  bureau’s  follow-up  investigations  more  efficient. 
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Another  area  where  there  is  room  for  improvement  concerns  the 
consistency  and  clarity  in  which  the  bureau  communicated  the  objectives 
of  the  Full  Count  Review  program  and  how  the  bureau  plarmed  to  use 
analysts’  input.  For  example,  materials  used  to  train  FSCPE  members 
noted  that  one  purpose  of  Full  Count  Review  was  to  document  issues  and 
“fix  what  can  be  fixed.”  However,  this  appears  to  be  inconsistent  with 
statements  made  by  bureau  officials,  who  noted  that  resolving  individual 
issues  was  beyond  the  scope  of  the  Full  Count  Review  program.  Moreover, 
according  to  one  bureau  official,  it  was  not  clear  internally  what  was 
meant  by  “fix  what  can  be  fixed.” 

None  of  the  bureau’s  documentation  or  training  manuals  that  we  reviewed 
explicitly  stated  that  the  bureau  would  only  check  for  systemic  errors. 
Because  of  the  inconsistent  message  on  the  purpose  of  the  Full  Count 
Review  program,  the  bureau  may  have  set  up  the  expectation  that  a  larger 
number  of  issues  would  be  resolved  during  Full  Count  Review.  For 
example,  one  FSCPE  member  told  us  that  he  expected  FSCPE  members 
would  identify  any  geographic  discrepancies  that  contrasted  with 
preliminary  census  data,  and  the  bureau  would  investigate  and  make  the 
necessary  changes.  He  noted  that  both  he  and  his  staff  were  very 
“dismayed”  to  find  out  that  certain  discrepancies  involving  group  quarters 
were  not  resolved  prior  to  the  release  of  the  public  law  data.  Another 
FSCPE  member  told  us  that  participants  were  strongly  motivated  by  the 
expectation  that  everything  would  be  done  to  correct  the  census  data. 


Conclusions 


The  Full  Count  Review  program  was  one  of  a  series  of  quality  assurance 
efforts  the  bureau  implemented  throughout  the  census  that  helped  ensure 
the  bureau  released  accurate  data.  Moreover,  FSCPE  members’ 
participation,  and  specifically  their  expertise  and  knowledge  of  local 
geography,  demographics,  and  housing  arrangements,  had  the  potential  to 
identify  data  issues  that  the  bureau  might  have  otherwise  missed. 

However,  the  fact  that  the  apportionment  and  redistricting  data  were 
released  with  around  4,800  unresolved  data  issues  of  unknown  validity, 
magnitude,  and  impact,  is  cause  for  concern,  and  indicates  that  the  bureau 
missed  an  opportunity  to  verify  and  possibly  improve  the  quality  of  the 
public  law  data.  Given  the  importance  of  accurate  census  data  and  the 
resources  that  bureau  staff  and  FSCPE  members  invest  in  the  Full  Count 
Review  program,  it  will  be  important  for  the  bureau  to  explore  how  to 
make  better  use  of  the  program  for  correcting  potential  errors  in  census 
data  in  the  future. 
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It  will  also  be  important  for  the  bureau  to  clarify  the  purpose  of  the  Full 
Count  Review  program  and  convey  that  purpose  clearly  and  consistently 
to  FSCPE  members.  Doing  so  could  help  ensure  that  the  bureau  meets 
FSCPE  members’  expectations. 


Recommendations 
Executive  Action 


To  help  ensure  the  accuracy  and  completeness  of  census  data  and  take  full 
advantage  of  the  Full  Count  Review  program  and  FSCPE  members’ 
participation,  we  recommend  that  the  Secretary  of  Commerce  direct  the 
bureau  to  develop  ways  to  resolve  a  larger  number  of  data  issues  prior  to 
the  release  of  the  public  law  data.  Specifically,  consideration  should  be 
given  to 


(1)  planning  the  Full  Count  Review  program  early  in  the  census  cycle  and 
testing  procedures  under  conditions  as  close  to  the  actual  census  as 
possible, 

(2)  integrating  the  Full  Count  Review  program  with  other  census 
organizational  units  and  operations  to  ensure  the  bureau  has  sufficient 
time  and  field  support  to  investigate  issues, 

(3)  developing  clear  guidelines  on  the  minimum  documentation  needed  for 
the  bureau  to  investigate  individual  data  issues, 

(4)  categorizing  issues  on  the  basis  of  the  quality  and  precision  of  the 
documentation,  and  investigating  first  those  issues  that  are  best 
documented  and  thus  more  easily  resolved,  and 

(5)  exploring  the  feasibility  of  using  staff  from  the  bureau’s  regional 
offices  to  help  investigate  data  issues  in  the  field  prior  to  the  release  of 
public  law  data. 

Moreover,  to  ensure  no  expectation  gaps  develop  between  the  bureau  and 
FSCPE  members,  the  Secretary  of  Commerce  should  also  ensure  that  the 
bureau  clarifies  and  consistently  communicates  to  participants  the 
objectives  of  the  Full  Count  Review  program  and  how  the  bureau  plans  to 
use  the  information  derived  from  it. 


Agency  Comments 
and  Our  Evaluation 


The  Secretary  of  Commerce  forwarded  written  comments  from  the  Bureau 
of  the  Census  on  a  draft  of  this  report  (see  app.  II).  The  bureau  concurred 
with  all  of  our  recommendations  and  had  no  comments  on  them.  The 
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bureau  also  provided  minor  technical  corrections  that  we  incorporated  in 
our  report  as  appropriate. 


As  agreed  with  your  office,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  from  its 
issue  date.  At  that  time,  we  will  send  copies  to  other  interested 
congressional  committees,  the  Secretary  of  Commerce,  and  the  Director  of 
the  Bureau  of  the  Census.  Copies  will  be  made  available  to  others  upon 
request.  In  addition,  the  report  will  be  available  at  no  charge  on  the  GAO 
Web  site  at  http://www.gao.gov.  Corirma  Wengryn,  Ty  Mitchell,  and 
Robert  Goldenkoff  made  major  contributions  to  this  report.  If  you  have 
any  questions  concerning  this  report,  please  contact  me  on  (202)  512-6806. 

/! 

Patricia  A.  Dalton 
Director 
Strategic  Issues 
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Appendix  I;  Disposition  of  Data  Issues 
Identified  During  Full  Count  Review  by  State 


Issues 

Total  accepted  for 

identified  by 

issues  identified 

research  by  Count 

FSCPE 

by  bureau 

Question  Resolution 

State 

members 

personnel 

program 

Alabama 

18 

54 

3 

Alaska 

21 

77 

4 

Arizona 

17 

57 

5 

Arkansas 

65 

72 

16 

California 

20 

71 

12 

Coiorado 

36 

29 

5 

Connecticut 

0 

70 

16 

Delaware 

0 

67 

0 

District  of  Coiumbia 

18 

61 

1 

Florida 

39 

61 

14 

Georgia 

23 

42 

6 

Hawaii 

0 

127 

5 

Idaho 

0 

62 

2 

Illinois 

112 

52 

15 

Indiana 

21 

55 

9 

Iowa 

71 

38 

21 

Kansas 

19 

46 

6 

Kentucky 

16 

60 

11 

Louisiana 

25 

66 

3 

Maine 

0 

67 

1 

Maryland 

8 

39 

0 

Massachusetts 

29 

38 

14 

Michigan 

23 

51 

32 

Minnesota 

69 

44 

30 

Mississippi 

0 

123 

1 

Missouri 

21 

37 

7 

Montana 

27 

69 

12 

Nebraska 

0 

93 

20 

Nevada 

29 

40 

5 

New  Hampshire 

0 

39 

0 

New  Jersey 

58 

52 

13 

New  Mexico 

32 

79 

4 

New  York 

33 

100 

33 

North  Carolina 

0 

149 

31 

North  Dakota 

19 

47 

11 

Ohio 

0 

174 

25 

Oklahoma 

14 

41 

13 

Oregon 

7 

57 

3 

Pennsylvania 

51 

56 

12 
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Appendix  I:  Disposition  of  Data  Issues 
Identified  During  Full  Count  Review  by  State 


State 

Issues 
identified  by 
FSCPE 
members 

Issues  identified 
by  bureau 
personnei 

Totai  accepted  for 
research  by  Count 
Question  Resoiution 
program 

Puerto  Rico 

20 

57 

18 

Rhode  Island 

25 

36 

7 

South  Carolina 

43 

51 

11 

South  Dakota 

33 

50 

11 

Tennessee 

31 

55 

7 

Texas 

86 

93 

1 

Utah 

19 

59 

8 

Vermont 

112 

35 

12 

Virginia 

0 

135 

8 

Washington 

59 

52 

8 

West  Virginia 

7 

59 

4 

Wisconsin 

0 

124 

11 

Wyoming 

26 

39 

10 

Total 

1,402 

3,407 

537 

Note:  The  table  includes  201  issues  that  were  placed  in  more  than  one  issue  category  (the  bureau 
had  five  categories  of  issues  depending  on  the  nature  of  the  discrepancy). 

Source:  GAO  analysis  of  U.S.  Census  Bureau  data. 
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Appendix  II:  Comments  From  the 
Department  of  Commerce 


THE  SECRETARY  OF  COMMERCE 

Washington,  D.C.  2Q23Q 


Ms.  Patricia  A.  Dalton 
Director,  Strategic  Issues 
U.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Ms.  Dalton: 

The  Department  of  Commerce  appreciates  the  opportunity  to  comment  on  the  General 
Accounting  Office  draft  report  entitled  2000  Census:  Refinements  to  Full  Count  Review 
Program  Could  Improve  Future  Data  Quality.  The  Department’s  comments  on  this  report  are 
enclosed. 


Warm  rega 


nara  Lrbvans 


Enclosure 
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Appendix  II:  Comments  From  the 
Department  of  Commerce 


Comments  from  the  U.S.  Department  of  Commerce 
U.S.  Census  Bureau 


U.S.  General  Accounting  Office  draft  report  entitled  2000  Census;  Refinements  to  Full 
Count  Review  Program  Could  Improve  Future  Data  Quality 

Comments  on  the  Text  of  the  Report 

1 .  Section:  Page  5,  last  paragraph,  continued  on  page  6  -  “The  Full  Count  Review  program 
ran  from  June  through  August,  2000.” 

Comment:  The  period  June  through  August  2000  was  the  training  phase  of  the 
Full  Count  Review  program.  The  program  ran  from  June  2000  through  March  2001 . 

2.  Section:  Page  6,  second  full  paragraph,  first  sentence  -  “The  bureau  contracted  with 
53  SCPE  members  who  reviewed  data  for  38  states,  the  District  of  Columbia,  and 
Puerto  Rico.” 

Comment:  The  53  FSCPE  members  represented  39  states  and  Puerto  Rico.  The  District 
of  Columbia  did  not  participate  in  the  Full  Count  Review  program. 

3.  Section:  Page  6,  second  full  paragraph,  second  sentence  -  “Bureau  employees  reviewed 
data  for  the  12  remaining  states  without  FSCPE  representation  in  Full  Count  Review.” 

Comment:  The  sentence  should  read  “. .  .  remaining  1 1  states  .  .  .  .” 

4.  Section:  Page  7,  first  paragraph  -  “Bureau  data  show  that  after  reviewing  census  data  for 
38  states,  the  District  of  Columbia,  and  Puerto  Rico,  FSCPE  members  identified  .  .  . .” 

Comment:  See  comment  in  response  to  Item  #2  above. 

5.  Section:  Page  7,  Table  1 

Comment:  In  the  GAO  analysis  in  Table  1,  201  of  the  issues  were  placed  in  more  than 
one  category.  For  this  reason,  the  Census  Bureau  cannot  independently  verify  the 
accuracy  of  these  data. 

6.  Section:  Page  9,  last  paragraph,  last  sentence  -  “In  all,  of  the  4,809  issued  [sic]  identified 
during  Full  Count  Review,  4,267  (89  percent),  were  not  subjected  to  any  further  review 
by  the  bureau  .  .  .  .” 
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Appendix  II:  Comments  From  the 
Department  of  Commerce 


2 

Comment:  The  1,457  issues  that  were  referred  to  the  Count  Question  Resolution  (CQR) 
program,  but  rejected,  were  “subjected  to  further  review,”  in  the  context  of  the 
CQR  program  criteria.  That  they  failed  to  meet  the  criteria  for  processing  under  the 
CQR,  and  no  further  action  was  taken  with  respect  to  these  issues,  does  not  mean  that  no 
additional  review  was  undertaken  after  they  were  identified  during  the  Full  Count  Review 
program. 

7.  Section:  Page  1 0,  third  paragraph,  last  sentence  -  “According  to  bureau  officials,  if 
funding  permits,  the  bureau  will  include  a  Full  Count  Review  program  as  part  of  a 
2004  test  of  census-taking  activities.” 

Comment:  The  Census  Bureau  is  still  developing  its  plan  for  the  2004  Census  Test  and 
has  not  determined  at  this  time  whether  it  will  include  a  Full  Count  Review  program. 
However,  we  certainly  plan  to  include  such  a  program  in  census  tests  later  in  the  decade. 

Responses  to  GAO  Recommendations 

Census  Bureau  Response:  The  Census  Bureau  concurs  with  the  recommendations  and  has  no 
specific  comments  on  them  at  this  time. 
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Appendix  III:  GAO  Products  on  the  Results 
of  the  2000  Census  and  Lessons  Learned  for  a 
More  Cost-Effective  Census  in  2010 


U.S.  General  Accounting  Office.  2000  Census:  Coverage  Evaluation 
Matching  Implemented  As  Planned,  but  Census  Bureau  Should  Evaluate 
Lessons  Learned.  GAO-02-297.  Washington,  D.C.:  March  14,  2002. 

U.S.  General  Accounting  Office.  2000  Census:  Best  Practices  and 
Lessons  Learned  for  a  More  Cost-Effective  Nonresponse  Eollow-Up.  GAO- 
02-196.  Washington,  D.C.:  February  11,  2002. 

U.S.  General  Accounting  Office.  2000  Census:  Coverage  Evaluation 
Interviewing  Overcame  Challenges,  but  Eurther  Research  Needed.  GAO- 
02-26.  Washington,  D.C.:  December  31,  2001. 

U.S.  General  Accounting  Office.  2000  Census:  Analysis  ofEiscal  Year 
2000  Budget  and  Internal  Control  Weaknesses  at  the  U.S.  Census 
Bureau.  GAO-02-30.  Washington,  D.C.:  December  28,  2001. 

U.S.  General  Accounting  Office.  2000  Census:  Significant  Increase  in 
Cost  Per  Housing  Unit  Compared  to  1990  Census.  GAO-02-31. 
Washington,  D.C.:  December  11,  2001. 

U.S.  General  Accounting  Office.  2000  Census:  Better  Productivity  Data 
Needed  for  Euture  Planning  and  Budgeting.  GAO-02-4.  Washington,  D.C.: 
October  4,  2001. 

U.S.  General  Accounting  Office.  2000  Census:  Review  of  Partnership 
Program  Highlights  Best  Practices  for  Euture  Operations.  GAO-0 1-579. 
Washington,  D.C.:  August  20,  2001. 

U.S.  General  Accounting  Office.  Decennial  Censuses:  Historical  Data  on 
Enumerator  Productivity  Are  Limited.  GAO-01-208R.  Washington,  D.C.: 
January  5,  2001. 

U.S.  General  Accounting  Office.  2000  Census:  Information  on  Short- 
and  Long-Eorm  Response  Rates.  GAO/GGD-00-127R.  Washington,  D.C.: 
June  7,  2000. 
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GAO’S  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accoimtability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  fimding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  the  Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full- 
text  files  of  current  reports  and  testimony  and  an  expanding  archive  of  older 
products.  The  Web  site  features  a  search  engine  to  help  you  locate  documents 
using  key  words  and  phrases.  You  can  print  these  documents  in  their  entirety, 
including  charts  and  other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  e-mail 
this  list  to  you  every  afternoon,  go  to  www.gao.gov  and  select  “Subscribe  to  daily 
E-mail  alert  for  newly  released  products”  imder  the  GAO  Reports  heading. 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

441  G  Street  NW,  Room  LM 

Washington,  D.C.  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Public  Affairs 

Jeff  Nelligan,  managing  director,  NelliganJ@gao.gov  (202)  512-4800 

U.S.  General  Accounting  Office,  441  G  Street  NW,  Room  7149 

Washington,  D.C.  20548 
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